rocksdb usage - Replace Column Size with more accurate Total Size#6540
rocksdb usage - Replace Column Size with more accurate Total Size#6540siladu merged 20 commits intobesu-eth:mainfrom
Conversation
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
|
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
siladu
force-pushed
the
add-blob-files-estimate-to-rocks-usage
branch
from
de5b82e to
f178a2e
Compare
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
…le-size Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
siladu
changed the title
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
…mate-to-rocks-usage Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
siladu
changed the title
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
|
hi. not directly related to your PR, but is the data you posted of a fully sync'd node? |
@sslivkoff No, this is synced with X_CHECKPOINT so missing block data since before October 2020 (the deposit contract deployment). |
good to know. thanks by chance do you have access to a full besu node that you could run your improved |
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
|
@sslivkoff here is one from another BONSAI node synced with X_SNAP which means it has all the blocks... Is that what you mean by full node, or do you mean a full archive node? (Note: the values may change depending on how much compacting rocksdb has done at the time of measurement. For example, restarting Besu triggers compaction and may reduce the DB size slightly) |
|
@sslivkoff by the way, if you're interested in state DB growth, this is a very relevant up and coming feature: https://wiki.hyperledger.org/display/BESU/Limit+Trie+Logs+for+Bonsai |
this is exactly what I was looking for. thanks! |
|
earlier the numbers were like this this seems like a more accurate count of the accounts (~235M) and storage slots(~1.1B) for ethereum mainnet. the newer number have 9.6M accounts and 23M slots afaict. do you know where this discrepancy comes from? |
I've updated the description to add a re-measured version of the checkpoint synced node back in.
I suspect it's because of ACCOUNT_INFO_STATE and ACCOUNT_STORAGE_STORAGE are part of the flat-db which is effectively a cache on top of the traditional TRIE_BRANCH_STORAGE. When this is enabled, we proactively populate the flat-db. Without it, it fills upon usage, so it will vary depending on how old your node is and what requests have hit it from peers for example. |
|
What is the expected output of --version ? I see different values between the one displayed by the subcommand and the one returned by web3_clientVersion |
There was a problem hiding this comment.
When using stats besu --data-path=/data/besu storage rocksdb stats. I see some metrics that don't exist, like : couldn't get property obsolete-sst-files-size.
I already shared some of them in the my comments below. We shouldn't have these warning messages.
|
|
||
| final PrintWriter out = spec.commandLine().getOut(); | ||
|
|
||
| final String dbPath = |
There was a problem hiding this comment.
I would suggest to rename parentCommand in RocksDbUsage, RocksDbSubCommand and StorageSubCommand to have a dbPath like this :
final String dbPath =
rocksDbSubCommand
.storageSubCommand
.besuCommand
.dataDir()
.toString()
.concat("/")
.concat(DATABASE_PATH);
There was a problem hiding this comment.
replace the .concat("/").concat(DATABASE_PATH); by .resolve(DATABASE_PATH)
| out.println("Column Family: " + getNameById(cfHandle.getName())); | ||
|
|
||
| final String prefix = "rocksdb."; | ||
| final String num_files_at_level_prefix = "num-files-at-level"; |
There was a problem hiding this comment.
I think this metric doesn't exist, you need to add the level number, like explain here
|
|
||
| final String prefix = "rocksdb."; | ||
| final String num_files_at_level_prefix = "num-files-at-level"; | ||
| final String compression_ratio_at_level_prefix = "compression-ratio-at-level"; |
| final String total_sst_files_size = "total-sst-files-size"; | ||
| final String live_sst_files_size = "live-sst-files-size"; | ||
| final String obsolete_sst_files_size = "obsolete-sst-files-size"; | ||
| final String live_sst_files_size_at_temperature = "live-sst-files-size-at-temperature"; |
There was a problem hiding this comment.
This metric is not exposed!
| final String num_running_flushes = "num-running-flushes"; | ||
| final String actual_delayed_write_rate = "actual-delayed-write-rate"; | ||
| final String is_write_stopped = "is-write-stopped"; | ||
| final String estimate_oldest_key_time = "estimate-oldest-key-time"; |
There was a problem hiding this comment.
It looks like this metric is not available when compaction is enabled, which is the case in Besu.
My approach was copy/paste all the available stats from rocksdb. I removed a few that were obvious duplicates and created unnecessary noise such as the sst dumps. There's more I could remove to tidy up. However, I am not 100% all the warnings will remain warnings forever, maybe they will become useful if we reconfigure rocksdb in the future? I also don't think there's much harm in the warnings. This subcommand is intended as an internal tool, unlikely useful to users. Maybe I should rename to |
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
…mate-to-rocks-usage Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
This is actually working fine, my besu command was referring to another version, the one defined in the PATH, which was different from the one that was running. |
|
I checked the results with the output displayed by speedb log-parser, and the output from this subcommand and the results are quite similar for the SST files, log-parser seems to not support yet blobs : This PR (besu --data-path=/data/besu storage rocksdb usage) Speedb log-parser |
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Remove nonsensical metrics Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
…mate-to-rocks-usage Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
@ahamlat easy, done. Also, renamed to "x-stats" Thanks for the feedback! |
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
…su-eth#6540) Add rocksdb x-stats command to print various stats for each column family --------- Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
4 participants
Replace "Column Size" with "Total Size" where Total Size = SST Files Size + Blob Files Size
besu --data-path=... storage rocksdb usagee.g. with a BONSAI + X_SNAP node...
BEFORE THIS PR
Column Size Total = 905 + 0.294 + 0.977 + 10 + 110 + 14 = 1042 GB
AFTER THIS PR
1104 GB is much closer to the on disk result:
Another example with a BONSAI + X_CHECKPOINT node:
Also dumping a bunch of stats out in...
besu --data-path=/data/besu storage rocksdb x-statswhich is useful for seeing the blobdb garbage stats (useful for #6289 (review)) among other things.
Example output for TRIE_LOG_STORAGE column family...